Computer vision assisted virtual reality calibration

A computer vision assisted semi-automatic virtual reality (VR) calibration technology has been developed that can accurately match a virtual environment of graphically simulated three-dimensional (3-D) models to the video images of the real task environment. In conventional model-based computer vision, camera calibration and object localization are performed sequentially. This sequential update cannot compensate for the inaccuracy in initial camera calibrations. We have developed a new 20variable weighted least-squares algorithm that updates both camera and object models simultaneously for given two camera views of two mating objects. This simultaneous update enables accurate model matching even with rough, approximate initial camera calibrations. The developed semi-automatic VR calibration supports automated intermediate updates, eliminating early all operator interaction except for initial coarse matching. In our quasi-static supervisory telerobotic applications, intermediate VR calibrations are performed intermittently at a few robot stopping poses only, as a cost-effective and safer approach compared to real-time visual servoing. Extensive experimental results comparing alignment errors under various viewing conditions are presented in the paper. Using the VR calibration technology developed, we have successfully demonstrated an orbital replacement unit (ORU) insertion task within